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Abstract While efTective concentration inequalities for suprema 
of empirical processes exist under boundedness or strict tail assump- 
tions, no comparable results have been available under considerably 
weaker assumptions. In this paper, we derive concentration inequal- 
ities assuming only low moments for an envelope of the empirical 
process. These concentration inequalities are beneficial even when 
the envelope is much larger than the single functions under consider- 
ation. 

1. Introduction. Powerful concentration and deviation inequalities for 
suprema of empirical processes have been derived during the last 20 years. 
These inequalities turned out to be crucial for example in the study of con- 
sistency and rates of convergence for many estimators. Unfortunately, the 
known inequalities are only valid for bounded empirical processes or under 
strict tail assumptions. So, this paper was prompted by the question whether 
useful inequalities can be obtained under considerably weaker assumptions. 

Let us first set the framework, starting with a brief summary of the known 
results for bounded empirical processes, or more precisely, for empirical pro- 
cesses index by bounded functions. To this end, we consider independent and 
identically distributed random variables Xi, Xn and a countable function 
class F such that sup^g^^r ||/||oo < 1 and supjgjr |E/(Xi)| = 0. The quantity 
of interest is then Y := sup^^gj- 1^ Y17=i Bousquet derives in [2] with 

cjy := supjgj- Var /(Xi) and v := ay + 2E,[Y] the exponential deviation 
inequality 



f[y - EY > V2^ + 3 j - ^ > °- 

Bousquet's proof is a refinement of Rio's proof in [8] and relies on the en- 
tropy method (see for example [7, Chapter 5.3]). Similar exponential bounds 



*The authors acknowledge partial financial support as members of the German-Swiss 
Research Group FOR916 (Statistical Regularization and Qualitative Constraints) with 
grant number 20PA20E- 134495/1. 

AMS 2000 subject classifications: Primary 60E15; secondary 60F10 
Keywords and phrases: Central Limit Theorem, Chaining, Concentration Inequalities, 
Deviation Inequalities, Empirical Processes, Rate of Convergence 

1 



2 



LEDERER ET AL. 



for bounded empirical processes have been found by Klein and Rio [5] and 
by Massart [6]. These bounds are slightly less sharp, but additionally hold 
for not necessarily identically distributed random variables and also for -Y. 
We finally mention Talagrand's work [10] that probably provided the spark 
for the development in this field. 

Results are also known for possibly unbounded empirical processes that 
have very weak tails. We consider independent and identically distributed 
random variables Xi, X„ and a function class J- that fulfills the Bernstein 
conditions, that is, supjg^ ^ ^"^^ E|/(Xi)r < ^K""'^, m = 2, 3, ... for a 
constant K. Additionally, we assume supj jgjr |E/(Xj)| = and card J-" = p. 
Biihlmann and van de Geer then derive in [3] the exponential deviation 
inequality 

pfy>/f. + V2^+./?i°iM+^MMUe-- forallx>0 

for Y as above. Other exponential bounds for unbounded empirical processes 
are given by Adamczak in [1] and by van de Geer and Lederer in [12]. These 
authors assume very weak tails with respect to suitable Orlicz norms. 

But what if the empirical process is unbounded and does not fulfill the 
strict tail assumptions mentioned above? There is no hope to derive expo- 
nential bounds as above under considerably weaker assumptions. However, 
we show in the following that weak moment assumptions are sufficient to 
obtain useful moment type concentration inequalities. For this purpose, we 
consider independent, not necessarily identically distributed random vari- 
ables Xi, ...,Xn and a countable function class with an envelope that has 
pth moment at most for ap G [1, oo). Our main result. Theorem 3.1, im- 
plies then for the quantity of interest Y := supjgjr |^ Yl^=i{f {■^i)~^f i-^i))]-! 
and for 1 < I < p, ay as above, and := (max{0, •})' 

E[y-(l + e)Ey]^< (^(^ + 7 + .)^ + l^y forane>0 
and 

We argue in Section 3 that this result is especially useful in the common 
case where the envelope (measured by M) is much larger than the single 
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functions (measured by cry). 



We close this section with a short outhne of the paper. In Section 2, 
we give the basic definitions and assumptions. In Section 3, we then state 
and discuss the main result. This is followed by complementary bounds in 
Section 4. Detailed proofs are finally given in Section 5. 



2. Random Vectors, Concentration Inequalities and Envelopes. 

We are mainly interested in the behavior of suprema of empirical processes 



(1) Y :- 



sup 



1 " 



i=\ 



or Y := sup 



1 " 

-5](/(X,)-E/(X,)) 



for large n. Here, Xi, are independent, not necessarily identically dis- 

tributed random variables and {/ : / S is a countable family of real, 
measurable functions. In the sequel, we may restrict ourselves to finitely 
many functions by virtue of the monotonous convergence theorem. 



Random vectors generalize the notion of empirical processes. Let Zi, 
be arbitrary probability spaces and {Zi{j) ■ Zi ^ M,l < j < N,l < 
i < n} a set of random variables. We then define the random vectors as 
Z{j) := {Zi{j), Zn{j))'^ : Z\ x ... x Zn — t- M". For convenience, we in- 
troduce their mean as PZ{j) := ^Y17=i^^i(^) ^^'^ their empirical mean 
as P„Z(j) := ^J2^=i ^iii)- Throughout this paper, we then consider the 
generalized formulation of (1) 

(2) Z := maxj¥nZ{j)\. 

The corresponding results for the empirical processes (1) can be found via 
Z,{j) := fj{Xi) or Zi{j) := fj{X,) - Efj{Xi) for T = {h, Jn}. 



Concentration inequalities are a standard tool to characterize the behav- 
ior of the process (2) (and thus of (1)). For n — oo, the process (2) is 
typically governed by the central limit theorem. In contrast, concentration 
inequalities bound the deviation in both directions from the mean or related 
quantities for finite n. Similarly, deviation inequalities bound the deviation 
in one direction only. Concentration or deviation inequalities - contrarily to 
maximal inequalities for example - provide bounds that depend only on n 
and moment properties of an envelope and the single functions / (and par- 
ticularly not on A^). 
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Let us finally express the moment properties of the envelope. First, we 
call £ := {£i, ...,£n)^ '■ Z\ X ... x Zn — s- M" an envelope if \Zi{j)\ < £i for all 
1 ^ J !^ and 1 < i < n . The basic assumption of this paper is then that 
there is a p G [1, oo) and an M > such that 

(3) Eff < MP 

for all 1 < f < n. Typically, the envelope is much larger than the single ran- 
dom vectors. In these cases, we have M ^ a := Y''maxi<j<Ar ^ '}2^=i 

3. Main Result. We are mainly concerned with concentration inequal- 
ities for unbounded empirical processes that only fulfill weak moment con- 
ditions. Additionally, we want to incorporate empirical processes with en- 
velopes that may be much larger than the single functions under considera- 
tion. 

The following theorem is the main result of this paper: 
Theorem 3.1. For I < I <p and e G R+ it holds that 

and, if additionally e G (0, 1], 

//86.4 \ IM A.jVlaV 

E|,l-.)EZ-Zt<(^(— + 7-.)-^ + — j . 

As discussed in the preceding section, we state our results in terms of ran- 
dom vectors instead of empirical processes. The connection can be made as 
described. Furthermore, we relinquished slightly better constants to obtain 
incisive bounds, however, a considerable improvement in / does not seem 
to be possible. We finally note that the expectation EZ can be replaced by 
suitable approximations. Such approximations are usually found with chain- 
ing and entropy (see for example [3], [13], [14]) or generic chaining (see for 
example [9], [11]). 

Let us now have a closer look at the above result. In contrast to the known 
results given in the introduction, the single functions may be unbounded and 
may only fulfill weak moment conditions. For the envelope, the moment re- 
strictions are increasing with increasing power I, as expected. 
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And what about large envelopes, that is M ^ cj? Theorem 3.1 separates 
the part including the size of the envelope (measured by M) from the part 
including the size of the single random vectors (measured by cr) . For p > 21 
and n ^ 1, a possibly large value of M is counterbalanced by — p-r ^ -7^ 

n P 

and thus, the influence of large envelopes is tempered. In particular, the 
term including the size of the envelope can be neglected for n — >■ 00 if p is 
sufficiently large. 



We conclude this section with two straightforward consequences of The- 
orem 3.1. 

Corollary 3.1. Theorem 3.1 directly implies probability bounds via 
Chebyshev's Inequality. Under the above assumptions it holds for x > 

' (Z > (1 + e)EZ + x)< min ^ " " 

i<i<p 



IM I 4.7^10- 



1_1 

-i P 



and similarly 

((«|l+7-.) 

P (Z < (1 - e)EZ -x)< min ^ 

\<\.<v 

Corollary 3.2. Concrete first order bounds under the above assump- 
tions are for example 

E\Z-2EZ], < 72^^ +4-^ 
j+ ' ^ 'n 



n p 

\^ 1 

and E 



-EZ- Z 
2 



M a 
<179.3 

1-^ \/n 



n p 



4. Complementary Bounds. In this section, we complement the main 
result Theorem 3.1 with two additional bounds. These additional bounds can 
be of interest if I is close to p. 



The first result reads: 

Theorem 4.1. Assume that the random variables Zi{j) are centered. 
For I > 1 and p > I it holds that 

I 

E[Z- 4EZ][^ - ^ ^ (0 (77) ' 
where T is the usual Gamma function. 
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Let us compare Theorem 4.1 with Theorem 3.1. On the one hand, the 
above result does not possess the flexibility of the factor (1 + e) and is a 
deviation inequality only. On the other hand, the term including the size of 
the envelope M is independent of p and has a different power of n in the 
denominator compared to the corresponding term in Theorem 3.1. Compar- 
ing these two terms in detail, we find that the bound of Theorem 4.1 may 
be sharper than the corresponding bound in Theorem 4.1 if / < p < 2/. 

We finally give explicit deviation inequalities for Z in the case of finitely 
many random vectors. For p > 2, explicit bounds are found immediately by 

replacing EZ in Theorem 3.1 or Theorem 4.1 by the upper bound ^8M5^M 
(see [4]). Another bound is found by an approach detailed in Section 5. The 
bound reads: 

Theorem 4.2. Let the random variables Zi{j) be centered. Then, for 
p>2, I £N, and p > I 

. ^ (-) 

This can supersede the bound in Theorem 4.1 for 2Y^iog(2iV) < 8\/2- 

5. Proofs. In this last section we give detailed proofs. 

5.1. Proof of Theorem 3.1. The key idea of our proofs is to introduce an 
appropriate truncation that depends on the envelope of the empirical pro- 
cess. This allows us to split the problem into two parts that can be treated 
separately: On the one hand, a part corresponding to a bounded empirical 
process that can be treated by convexity arguments and Massart's results on 
bounded random vectors [6]. And on the other hand, a part corresponding 
to an unbounded empirical process that can be treated by rather elementary 
means. 

For ease of exposition we present some convenient notation for the trun- 
cation first. After deriving two simple auxiliary results, we then turn to 
the main task of this section: We prove Lemma 5.3, a generalization of 
Theorem 3.1. The main result of this paper. Theorem 3.1, is then an easy 
consequence. 

A basic tool used in this section is truncation. Before turning to the proofs, 
we want to give some additional notation for this tool. First, we define the 



E 



Z -2M 



log(2iV) 



n 
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unbounded and the bounded part of the random vectors as 



Z{j) := {Zi{j),...,Zn{j)V 
m ■■= {Z,{j),...,Z^{j)f 

Similarly, we define 



{Zl{j)l{£^<K}, Zn{j)l{e„<K}V ■ 

{£l{3)'^{£i>K},-,£n{j)l{Sn>K}f 
{£l{j)'^{£i<K}, ■■■,£n{j)l{£„<K}f ■ 



To prevent an overflow of indices, the truncation level K > is not included 
explicitly in the notation. The truncation level is, however, given at the 
adequate places so that there should not be any confusion. Finally, we define 
the maxima of the truncated random variables as 

Z := max |P„Z(7')| and Z := max |P„Z(7)|. 
Now we derive two simple auxiliary lemmas. 

Lemma 5.1. Let I > 1, Wi : Zi for 1 < i < n and EW^ < 1. 

Then, 

E [¥nWf < 1 

for the corresponding random vector W on the product space. 
Proof. By the triangle inequality we have 



l\ I 




i=l 



< 



1 



n 



i=l 



w 



< 1. 



□ 



Lemma 5.2. Under the assumptions of Theorem 3.1 it holds for K 



UP M that 



and a< a for 



\E[Z- Z]\ < 



M 



'(1--) 
n p 



a := 



1 " 

max — > YaiZAi). 



i<j<N n ^ 



4 = 1 
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Proof. Since llal 



< |a — 6| for all a, 6 G M, it holds that 



|E[Z- Z]| 



E 



max lP„Z(j)| - max lP„Z(j)| 



< E 

< E 
= E 

< E 



max r„Z(j)| 

l<j<A' 



Z{j)\\ 



max |P„(Z(j)-Z(j))| 

l<j<A' 



max r 

■ n 



i=l 



With Holder's and Chebyshev's Inequality we obtain for I < i < n 



<(Eff)p(El|^^>^|)^"p 



<(Eff)^ 



KP 



<- 



MP 



-I ■ 



These two results yield then the first assertion. The second assertion is 
straightforward. □ 

We can now turn to the harder part of this section. The following lemma 
is a generalization of Theorem 3.1. The derivation of Theorem 3.1 from this 
and Lemma 5.2 is then a simple task. 

Lemma 5.3. Letl<l<p and e, K G M+. Then, 



E Z - 1 + e)EZf. < — + 5 — + + 



and 



E[(l - e)EZ- Zf < 



86.4 \ IK A.lVla 

+ 5 — + + — 

e J n Vn Ki 



Ml \ 
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Proof. The key idea of the proof is to separate the bounded from the 
unbounded quantities. On the one hand, we develop bounds for EZ^ via ele- 
mentary means. On the other hand, we develop bounds for E [Z — (1 + e)EZ]'j_ 
and E [(1 — e)E_Z — zf_^ via convexity arguments and [6, Theorem 4]. They 
can then be combined to deduce the result. 

We start with the proof of the first inequality. First, we split Z in a bounded 
and an unbounded part 

Z= m^ax^|P,,Z(j)| 

= max |P„(Z(i) + Z(j))| 

1<J<N 

< max (|P„Z(j)| + |P„Z(j)|) 

l<j<A' 

< z + z 

and deduce with the triangle inequality that 
E [Z - (1 + e)EZ]'^ 

<E [Z + Z- (1 + e)EZ]'_^ 

<E [(Z-(l + e)EZ)+ + Z+]' 
(4) < ((E[Z-(l + e)EZ]^)T + (Ez'^)Ty . 

Now, we turn to the development of bounds for Ez!)_. With the help of 
Holder's and Chebyshev's Inequalities we obtain as above for 1 <i <n 

MP 

E£] <- 



That is, 



E 



I 



< 1. 



We may consequently apply Lemma 5.1 and obtain 

^-^i ^ MP 



(5) EZ+ = EZ < E[P„^]' < . 

As a next step, we derive bounds for E [Z — (1 + e)EZ]'_j_. To begin, we set 
J := + 2.5)K, a := a and define the function gi : M+ — )• (1, oo) as 

gi{x) := exp ( 
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Note that gi is strictly increasing and smooth. Moreover, the first and second 
derivatives are 



2n(V2cj2 + JxT - V2<t) J 
9i{x)= , -jxi gi{x) 



n 
'21J 



V2a 



20-2 + JxT 



XI gi{x) 



and 



n J 2_2 
^ 1 

1 . 3 



2^</ 2(20-2 _^ 



I 2 



- — 2 / \ 

gi{x) 




\/2(T \ A 



2o-2 + JxT 



2 




1 



y - 1 ) ^c/;(a;) 



^nxT-g,{x)(_ !!+2(l-/).-0. 

We now use the lower bound for the second derivative to find an interval, 
on which the function gi is convex. To this end, we observe that for cj > 

\/2a \ n , . 

J + 2(1 - l)x-- > 




2o-2 + JxT 



_J: ^ 2(/-l)J _i 

= > X I . 

This can be rewritten with the definition u := tt^xT > as 



1 ^ (/ - 1) J 



2 



2 



and with the definition C := > as 



(7) l-^>-- 
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We assume now, that u> C. Then, 



1- ' >^ 

^ Vl + u ( 1 - — ) > 1 



u 

^ {l + u){u^ -2Cu + C^)>u^ 

^ - 2Cu^ + (C^ - 2C)u + C^>0. 

Considering the equahty 

-2Cu + C^-2C = 

with roots {C it \/2C}, we deduce that 

- 2Cu^ + (C^ - 2C)u + >0 

for ah n > C + y/2C. Consequently, for u > C + V2C, Inequahty (7) holds 
true. Hence, if we postulate 



J 1 (I- 1) J2 12(1 - 1) J2 
(8) TT-^^^ > '—-^ +■' ' 



2cj2 ncj2 V ncr^ 

Equation (6) holds true. The postulate (8) is equivalent to 



1 2(1 - 1) J J8(l - 1)0- 

XT > h — 



n y n 

and to 

\ n y/n J 

Additionally, note that with this condition on x. Equation (6) is also true 

2 _2 

for cr = 0. So we finally derived, since (-*- / i ) is positive, 

that the function gi is convex on the domain (/, oo). 

This convexity property makes it possible to apply a result of [6]. To show 
this, we introduce 

X -.= {^-{1 + e)EZ)'^_ 
and find with Jensen's Inequality and the fact that gi is increasing 

5/(EX) < gi{¥.[X V /]) < ^gi{X V /). 
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We used here the notation aV b := max{a,b} for a, 6 G M. Massart's In- 
equahty [6, Theorem 4, (13)] for bounded random vectors translates then to 
our setting as 

¥{nZ > (1 + e)nEZ + aVSmii + + 2-5^ Kx) < e""^, 
where e, x > 0. This is equivalent to 



(9) 



Rr T 

F(Z > (1 + e)EZ + a\ — + -x) < e"^. 

V n n 



We then deduce (cf. [12]) 



Eexp 



( ( \/2(j2 + J(X VI)T -\/2cj )\ 



V2J 



/ 



exp 

V V 



V2J 



> t 



dt 



<1 + 



=1 + 



exp 

V V 



/ J 



V2J 



> t 
I J 



dt 



2o-2 + J{X V I)T > ^/2cr + ^/ =^ log t 1 dt 



2J2 



=1 + 
and note that 



J{X\/ 1)1 > 4crW — logt + — logt ) dt 
n n I 



1 / J2 2 J2 

J/! < 4o-\/ — logtH logt 

n n 



2(1 - 1) J Jsir^a , 
^ — — + ^ ^ ^ ' < 4cj 



n 



n 



logt 2J 
^ log t. 



n n 
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Eexp 



^^^^2cj2 + J(X V/)T -V2(J^^ 



V2J 



<l + e'"^ - 1 + 



^1 , /logt 2J, \ , 
XT > 4a\l-^ + — lost dt 



n n 



P > (l + e)EZ + 4cr 
<e'"^+/ exp(-logt2)(it < e'. 



H log t (it 



n n 



In summary, we have 

gi{¥.X) < eK 

This is now inverted, observing that for y G (1, oo) such that 



y = exp 



Vn(V2cr2 + JxT - \/2cr) 

72J 



ylog 



^/2J 

/o T I 

A/log y + \/2c7 = Y 20-2 + JxT 



2J 



4(TJ r ^ 1 

logy + —i=\J\o%y = Jxi 

/2J, 4(7 V 

This is now applied with x = EX to obtain 



(10) 



EX < 



2/ J 4.y/la 
n \fn 



We are now ready to collect the terms. Inequalities (4), (5) and (10) give 

w[7 n ^ (2U \^la Ml y 
E Z - 1 + e EZ^ = + + . 
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Hence, since J = + 2.5)K, 

This finishes the proof of the first part of the lemma. For the second part, 
we note that 

Z= max^|P„ZO-)| 

= max |P„((Z(j)-^(j))l 

l<j<A' 

< max (|P„Z(j)| + |P„Z(j)|) 

l<j<A' 

<z + z 

and therefore Z > Z^ — Z. Consequently, 
E[(l - e)EZ- Zf_^_ 
<E [(1 - e)EZ- Z + 
<E [((l-e)EZ-Z)+ + Z+]' 
= (E[{l-e)EZ-Z]\)T + (EZ+)Ty . 

One can then proceed as in the first part and use [6, Theorem 4, (14)]. □ 

Proof of Theorem 3.1. Set if = n^M in Lemma 5.3 and use Lemma 5.2 
to replace the truncated quantities by the original ones. □ 

5.2. Proof of Theorem 4-1- Here, we prove Theorem 4.1 with the help of 
symmetrization and Massart [6] . The proof here is considerably shorter than 
the proof of Theorem 3.1. This is because we do not need tedious convexity 
arguments. 

Proof of Theorem 4.1. The trick is to use symmetrization and desym- 
metrization arguments so that we are able to use [6, Theorem 9] in a favor- 
able way. 

Beforehand, we define := niaxi<j<Ar | ^ ^"^^^ ejZj(j)| with independent 
Rademacher random variables e^. Then, we symmetrize according to [13, 
Lemma 2.3.6] with the function ^(x) = (x — 4EZ)f)_ to obtain 

E[Z- 4EZf, < E [2Z, - 4EZf, 
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and we desymmetrize with the function <I>(x) = x to obtain 
E [2Z, - 4EZ]^|_ < E [2Z, - E2Z,]^^ . 

Hence, 

(11) E [Z - 4EZ]'_^ < 2'EEe [Z^ - ¥.Z^]\ , 

where we write here and in the following E^ for the expectation and for 
the probability w.r.t. the Rademacher random variables. Next, we observe 
that 



Ee [Z, - EZ,]', 

= p¥,[{z,-¥.z,)\>t)dt 

oo 

> EZ, + tT ) dt 



e 1 -^e 



r-oo 11 1 1 \ 

< / PJ max - VeiZi(j) >E max - VeiZi(j)+t' dt 

+ / PJ max — y^ejZi(?) > E max -^^ eiZi(j) + n \ c 
Jo \i<j<N i<j<Nn^^ ] 

(1 " 1 " i\ 

max - eiZAj) > E max - eiZAj) + (it. 
i=l 1=1 / 



<2 

In a final step, we apply Massart's Inequality [6, Theorem 9] with 

n 

= ^rnax^^ (2|Zi(j)|)2 < 4nP„£:2, 

~ ~ i=l 

where P„f2 i ^n^^ _r2. This yields 
2 



(1 " 1 " i\ 

max — > eiZAj) >E max — > eiZi(j)+t~ dt 
i=l i=l / 



oo 



<2 / exp ( — I 



£2 



n 



2 ,_ ' 



2 - (P„£:^)2 / exp(-tT]dt 



n / 2 
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With Inequality (11) this gives 



E [Z - 4EZ]'^ < 2'/ ( - ) E [p„£:2] 2 r 





Finally, because of Lemma 5.1, it holds that 



E [P^^'^] 2 < E [P„f]' < 



and hence 



E [Z - 4EZf < IT (-] (—] ' mK 




□ 



5.3. Proof of Theorem 4-2. We eventually derive Theorem 4.2 using trun- 
cation. After some auxiliary results, we derive Lemma 5.6. This Lemma set- 
tles the bounded part of the problem. It is then used to proof Lemma 5.7 
which is a slight generalization of the main theorem. Finally, we derive The- 
orem 4.2 as a simple corollary. 

We begin with two auxiliary lemmas: 

Lemma 5.4. Let W be a centered random variable with values in [—A, A], 
A>0, such that EW^ < 1. Then, 



w 1 

Ee~ < 1 + -TTT. 



Proof. We follow well known ideas (see e.g. [3, Chapter 14]): 



w w W 

Ee^ = 1 + E e^-1 

A 



<1+E e— - 1 - 



\W\] 



A 




00 



m=2 




m=2 




□ 
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Lemma 5.5. LetC:^ := \{{h,...,imf G {1, n}™ : Vj G {l,...,m}3/ G 

{1, 7^ j, = for m, n G N. Then, 



Proof. The proof of this lemma is a simple counting exercise. We start 
with the case m < 2. One finds easily that Cf = and C2 = n, which 
completes the case m < 2. Next, we consider the case m > 2. To this end, 
we note that = 1, C| = 2 and < 2™ < m! for m > 3. This completes 
the cases n < 2. Now, we do an induction in n. So we let n > 2 and find 



n+l 



mim 



1] 



2! 



a 



m-2 



+ 



m(m- l)(m- 2) 



3! 



a 



m—3 



+ ... + 



m{m — 1)...3 

(m-2)! 



+ 1. 



By induction, this yields 



<m! 



I m I 



-I I 771 — 2 I 



^ 2! V2 



+ 



-I , I m — 3 I 



3! V2 

We now assume that m is even. So 



+ ... + 



(m-2)! V2 



+ 1. 



<m! 



=m! 



<m! 



2/ 2!V2. 



)m 1 
^ 2! V2 



1 (n\ 2 
3! V2 



m 1 

2 ~^ 



+ ... + 



1 



n 



i=2 



1 



+ 



(m-2)! V2 
1 \ /"n\f-3 



+ 1 



(2i-l)! (2j)! 



+ 1 



m T 2 / 777, \ / -I 



i=2 



2 \2) \2 



2 

'E 

i=o 



2 \ / 1 



2. 



mi 



n + l 



This completes the proof for m > 2 with m even. We note finally, that for 
odd m > 2 we have < mC'^_^ < ml (§) . □ 

We now settle the bounded part of the problem. Bounded random vari- 
ables are in particular subexponential, so one could apply results from [15] 
for example. But for our purposes, a direct treament as in the following 
seems to be more suitable. 
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Lemma 5.6. Let / e N, p > 2 and A> 2. Then, for the truncation level 
K = ^ + Jf^l, 



E 



max (Fn - P)Z(j) - AM ^°^^^^ 



A n j ' 



Proof. We assume w.l.o.g. M = 1 and observe that 

^[Mj)-mij)f<^Mj?<^- 



Moreover, because of Holder's and Chebyshev's Inequalities and A' > 1 it 
holds that 

\Z,ij) - EZ,(i)| < \Z,ij)\ + \KZ,ij)\ <K + ^ = A. 

These observations, the independence of the random variables and Lemma 5.4 
yield then 

^ n(Pn-P)Z(j) 

Ee A 
=Ee A 



< 1 + 



A^ 



Next, one checks easily, that the map x i— e^' is convex on the set [{I 
l)',oo). Hence, using Jensen's Inequality again, we obtain 



E 



max (P„ - P)Z(j) - 

i<i<Af n 



<^E 

n' 



<- 



n' 



A 



n' 



max^ n{Fn - P)Z_{j) /A - log(iV) ) V (/ - 1)' 



<4- log' ( Eexp ( ( max n(P„ - P)Z(i)/A - log(iV) ) V - 1) 
_A 

TV 

A 



log' ( Eexp ( ( ^max^?i(P„ - P)Z(j)/^ - log(iV) ) V (/ - 1) 



log' max Eexp(n(Pn - P)Z(j)M) + e' 



A^ 
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We finally note that a + b < eab for all a, 6 > 1 and find 



E 



max I 



<— log 



P)Zij) - A 



1 



log(iV) 



n 



1 + 12' - 



log(l + -^) +loge' 



< 



A n I 



□ 



The results above can now be used to derive a generalization of the main 
problem. 



Lemma 5.7. Assume that the random variables Zi{j) are centered. Then, 
for I en, p>2 , p> I and A > 2, 

I 



E 



Z -AM 



log(2iV) 



n 



< 2 



A 



p-i 



2 1 lA 

n A n 



Proof. The idea is again to separate the bounded and the unbounded 
quantities. The part with the unbounded quantities is treated by elementary 
means and Lemma 5.5. For the bounded part, we use Lemma 5.6. 

1. Then, we 



A2 



First, we assume w.l.o.g. that M = I and set K = ^ + , , ^ 
deduce with the triangle inequality that 

E 



max FnZ(j) - ^i^^^M 



=E 



<E 



max (P„ - P)Z(j) - 



— loefiV) 
max (F„ - P)Z(j) + max (P„ - P)Z(j) - A ' 



<E 
(12) 

< I ( E 



max (P„ - P)Zii ) 



+ 



max I 



log(iV) 



n 



max I 

l<j<N 



+ E 



log(iV) 



^max^(P„ - P)Z{j) - 
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So, we are able to treat the unbounded and the bounded quantities sepa- 
rately. We begin with the unbounded quantities. We first note that 



[(P„-P)z(j)]' <((P„ + P)£:)' 



n-P)£ + 2P£y 



-I I 



max (P„ - P)Z{j) 
i<j<p 



< (2P£+{K [{F^-P)£]')T^' 



Hence, 
(13) E 

Holder's and Chebyshev's Inequalities are then used to find 

1 " 1 " 1 



i=l i=l 

To bound the quantity left, we note that for all i and p > q G'N 

E [£i-E£iY < 21 

so that 

E[(£i, -E£,,)...(£i^ -E£,^)] < 2'. 

Moreover, it holds that 

E[{£i,-E£,,)...(£i,-E£i,)] =0 

for all such that there is a j with ij ^ iji for all j' ^ j. With 

Lemma 5.5, we then get for n > 1 



(15) 



2^01" 2'/!/n\LiJ 



E[(P„-P)^' < ^ < 



n' V2 



< II 



Clearly, this also holds for n = 1 and 1 = 1. For n = 1 and / > 1 we note 
that 



E [(p„ - p)£]^ < 2' < nW, 



so that Inequality (15) holds for all n and / under consideration. Inserting 
then Inequalities (14) and (15) in Inequality (13), we obtain the result for 
the unbounded part 



(16) 



E 



max (P„ - P)Z{j) 

i<j<p 



< 



KP 
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Next, we plug the result of Lemma 5.6 and Inequality (16) in Inequality (12) 
to derive 



E 



max FnZ(j) - 




^ ^ \ n A n 



Finally, we define Z{j + A^) := -Z{j) for 1 < j < iV. We then get 



E 



A 



log(2iV) 



n 



=E 



max P„Z(?) - A 

l<j<2Af 



log(27V) 



< 2 



A 



n 



+ 



„„i /2 1 lA 
^ ' V n A n 



replacing by 2N in the results above. 



□ 



Theorem 4.2 is now a simple corollary. 



Proof of Theorem 4.2. Set A = 2^/n in Lemma 5.7. 



□ 
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